Each custom severity threshold key requires both a corresponding icon and color. Without them, plugin will not function as YAML configuration is validated on startup.

It would make sense to add links to the prerequisites,
e.g Install scorecards: https://docs.redhat.com/en/documentation/red_hat_developer_hub/latest/html-single/evaluate_project_health_using_scorecards/index#enable-scorecards_set-up-scorecards-to-monitor-your-project-health
And you also need at least one of the modules (Jira, GitHub, OpenSSF), e.g. https://docs.redhat.com/en/documentation/red_hat_developer_hub/latest/html-single/evaluate_project_health_using_scorecards/index#install-and-configure-scorecards-to-view-metrics_evaluate-project-health-using-scorecards

2. You have {configuring-book-link}[added a custom {product-short} application configuration].
3. An existing default Scorecard plugin configuration is deployed with at least one Scorecard plugin module, see https://docs.redhat.com/en/documentation/red_hat_developer_hub/latest/html-single/evaluate_project_health_using_scorecards/index#install-and-configure-scorecards-to-view-metrics_evaluate-project-health-using-scorecards.

We're not allowed to have links in the prerequisites section. I added an additional resources section at the end of the procedure. As we move to DITA we will have more rigid structural requirements and restrictions.

Should we use app-config.yaml to be consistent with naming of this configuration file in the rest of the Scorecard docs?

done

It would be good to mention somewhere what are the options for color configuration:

Color configuration - supports multiple formats:

• theme palette reference (success.main / warning.main / error.main)
• HEX code (e.g. '#FFA500')
• RGB/RGBA (e.g. 'rgb(255, 0, 0)')
  Threshold rules 'success', 'warning' and 'error' have default colors ('success.main', 'warning.main', 'error.main'.

The bullet point with: "- theme palette reference (success.main / warning.main / error.main)" means that administrators can configure custom severity threshold key also with default colors, e.g.:

            - key: error
              expression: '>50'
              color: error.main
              icon: scorecardErrorStatusIcon

I created a reference topic to capture this information.

It would be good to use as examples icons that work in rhdh by default (check and error are not ingested in rhdh, so they won't appear):

scorecard:
  plugins:
    myDatasource:
      myMetric:
        thresholds:
          rules:
            - key: ideal
              expression: "<10"
              color: "#5CE65C"
              icon: star
            - key: warning
              expression: 10-50
              color: "rgb(233, 213, 2)"
              icon: monitor
            - key: critical
              expression: ">50"
              color: error.main
              icon: scorecardErrorStatusIcon

I have updated icons to the ones that will work. I have updated colors to be clearly different from the default ones (default dark green is now light green, default orange is now yellow). I have updated keys to be the custom ones instead of default success, warning, error. The critical key shows default color and icon example.

It would be also great if you could mention information about what icons are accepted:
Icon configuration - supports multiple formats:

• Backstage system icons: 'kind:component', 'kind:api', etc.
• Material Design icons: 'settings', 'home', 'build', etc. = NOTE that from these, only the ones that are ingested in RHDH actually work - e.g check doesn't work
• SVG strings: '...'
• URLs: 'https://example.com/icon.png', '/assets/icon.svg'
• Data URIs: 'data:image/svg+xml;base64,...'

Threshold rules 'success', 'warning' and 'error' have default icons: scorecardSuccess, scorecardWarningStatusIcon, scorecardErrorStatusIcon

I don't know if there is already page about icons in RHDH, since the same logic for providing icons is also for header I think.

I updated the YAML example and added some information in the reference topic.